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Summary 

Assessing  the  effectiveness  of  an  ongoing  military  humanitarian  assis¬ 
tance  operation  (HAO)  is  difficult.  These  operations  tend  to  be  less 
familiar  to  the  military;  the  operation’s  objectives  are  often  vague; 
and  the  initial  available  information  on  a  humanitarian  crisis  is  often 
incomplete  or  inconsistent.  But  such  an  assessment  is  important  to 
provide  insight  into  what  strategies  are  working;  where  assets  need  to 
be  shifted;  whether  more  (or  fewer)  forces  are  required  for  specific 
tasks  or  the  mission  as  a  whole;  and  when  the  end  of  the  operation 
(end  state)  has  been  reached. 

To  help  solve  the  problem  of  assessing  effectiveness  in  HAOs,  a  mili¬ 
tary  command  can  use  a  variety  of  analytical  tools,  including  mea¬ 
sures  of  effectiveness  (MOEs) .  Such  measures  are  used  in  combat 
operations  as  indicators  of  an  operation’s  progress.  However,  in  past 
HAOs  when  measures  were  developed  to  track  progress,  they  may  not 
have  been  thought  through  clearly.  Such  measures  were  often  devel¬ 
oped  very  quickly,  with  little  or  no  opportunity  to  develop  a  compre¬ 
hensive  package  or  to  test  the  appropriateness  of  individual  measures 
before  putting  them  into  use.  Also,  many  confuse  any  quantitative 
information  with  MOEs.  MOEs  measure  effectiveness;  other  statistics 
measure  other  things  that  can  add  to  a  picture  of  an  HAO,  but  they 
do  not  measure  effectiveness. 

It  is  important  to  put  analytical  tools  in  the  proper  context  They  are 
one  way  of  determining  information  requirements,  analyzing  infor¬ 
mation,  and  organizing  the  results  so  that  they  are  understandable 
and  useful.  These  tools  help  a  commander  to  gain  a  picture  of  the 
operation  that  enables  him  to  make  more-informed  decisions.  But 
they  should  not  be  an  end  in  themselves  or  a  substitute  for  a  com¬ 
mander’s  judgment 

This  paper  uses  lessons  from  past  operations,  exercises,  and  studies  to 
contribute  to  the  military’s  understanding  of  how  to  assess  progress 
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in  an  HAO.  We  discuss  not  only  MOEs,  but  other  measures  that  are 
useful.  This  analysis  is  provided  in  connection  with  a  Center  for  Naval 
Analyses  (CNA)  study  intended  to  help  the  Marine  Corps  improve  its 
ability  to  conduct  HAOs  [1-13].  The  Marine  Corps  Combat  Devel¬ 
opment  Command  (MCCDC)  and  I  Marine  Expeditionary  Force 
(I  MEF)  sponsored  this  study. 

Key  findings 

We’ve  identified  five  types  of  measures: 

•  Level-of-effort  measures  describe  the  magnitude  of  specific  efforts. 

•  Task-performance  MOEs  provide  additional  insight  into  how  well 
military  and  humanitarian  tasks  are  proceeding  in  support  of 
the  HAO. 

•  Mission-level  MOEs1  provide  insight  into  progress  toward  the 
larger  political  objectives. 

•  Transition  measures  provide  insight  into  progress  toward  the 
transition  of  responsibilities  to  another  force  or  organization. 

•  General  indicators  provide  insight  into  progress  on  improving 
the  situation.  Operational  indicators  are  a  non-quantitative 
tool — a  supplement  to  MOEs — that  can  “indicate”  progress. 

These  different  types  of  measures  build  upon  each  other  and  should 
be  used  together  to  get  a  more  comprehensive  view  of  the  situation 
the  military  is  trying  to  measure.  The  first  three  types  of  measures  par¬ 
allel  the  mission-analysis  process:  Political  and  policy  level  objectives 
are  addressed  through  mission-level  MOEs;  from  these  objectives, 
tasks  are  derived,  and  these  are  addressed  through  task-performance 
MOEs;  and  from  the  tasks,  required  capabilities  and  forces  are 
derived,  and  their  actions  are  addressed  by  level-of-effort  measures. 


1.  These  MOEs  measure  progress  toward  the  larger  political  and  policy 
objectives  set  forth  for  the  mission.  The  mission  will  encompass  a  variety 
of  military  and  humanitarian  tasks,  which  are  measured  by  task-perfor¬ 
mance  MOEs. 
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These  types  of  measures  are  part  of  a  framework  for  developing  a 
comprehensive  set  of  measures  for  HAOs. 

Based  on  our  analysis  of  past  operations,  we  found  that  HAOs  gener¬ 
ally  involve  some  or  all  of  the  following  components:  diplomacy,  secu¬ 
rity,  infrastructure  and  institutions,  public  health,  and  the  agriculture  and 
economic  situation. 

Diplomacy  is  not  discussed  in  this  paper,  as  our  focus  here  is  on  oper¬ 
ations.  However,  military  and  humanitarian  actions  should  be  aligned 
with  ongoing  diplomatic  efforts. 

These  HAO  components  and  the  types  of  measures  used  to  address 
them  provide  a  framework  for  developing  measures  to  assess  effec¬ 
tiveness.  However,  measures  within  this  framework  should  be  exam¬ 
ined  as  a  whole  because  these  components  and  types  of  measures  are 
inextricably  linked.  For  example,  actions  taken  to  support  identified 
security  tasks  may  improve  the  public-health  situation.  Relief-organi¬ 
zation  efforts  to  increase  food  distribution  may  improve  the  security 
situation,  and  ultimately  these  efforts  may  be  reflected  in  security  and 
public  health  mission-level  MOEs. 

MOEs  must  be  carefully  chosen.  Table  1  summarizes  criteria  for  MOE 
development. 

MOEs  must  be  used  appropriately.  Potential  problems  with  using 
MOEs  tire  as  follows: 

•  If  not  carefully  chosen,  the  cost  in  time  and  effort  for  collection 
and  analysis  of  MOE  data  may  outweigh  the  benefits. 

•  One  pitfall  is  that  a  single  MOE  may  come  to  be  viewed  as  the 
key  to  success,  rather  than  as  a  possible  indicator  of  it. 

•  Factors  outside  the  military’s  control,  such  as  the  weather  or  the 
effectiveness  of  other  militaries,  the  host  nation,  relief  organi¬ 
zations,  and  so  forth,  influence  overall  effectiveness  in  an  HAO, 
and  this  will  be  reflected  by  the  MOEs. 

With  these  problems  in  mind,  a  commander  needs  to  place  the 
proper  emphasis  on  measures  and  put  them  in  the  proper  context.  In 
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addition,  he  must  ensure  that  they  are  used  correctly.  Measures  are  a 
tool  for  understanding  the  humanitarian  situation  and  how  the  mili¬ 
tary  is  affecting  this  situation,  and  improving  military  effectiveness. 


Table  1 .  Criteria  for  choosing  analytical  measures 


Criteria  Definition 


Mission-related 

i 

_ 

Mission-level  MOEs  should  relate  to  the  political  objectives  set  forth  for  the  overall  j 
mission.  The  other  MOEs  and  measures  will  be  more  specifically  focused,  but  must  j 
still  maintain  a  connection  to  the  mission's  objectives. 

Comprehensive 

The  full  list  of  task-performance  MOEs  should  coverall  tasks  in  support  of  the  mission  ! 
and  should  expand  if  the  mission  expands.  j 

Meaningful 

i 

Mission-level  MOEs  should  focus  on  the  effectiveness  of  the  overall  mission,  task-per¬ 
formance  MOEs  should  focus  on  the  effectiveness  of  tasks,  not  their  accomplishment, 
and  transition  MOEs  should  focus  on  the  degree  that  the  tasks  are  transitioning  to  the  j 
follow-on  organization.  They  should  all  be  evaluated  together. 

I  Measurable 

i 

Those  managing  the  analytical  measure  effort  must  give  clear  guidelines  so  that  infor- : 
mation  can  be  collected  and  measured  consistently  over  time  and  across  areas. 

Sensitive 

i 

The  level  of  MOE  measurement  should  provide  enough  detail  to  ascertain  whether 
the  situation  is  changing.  MOEs  should  support  trend  analysis. 

!  Timely 

Mission-level  MOEs  should  be  responsive  to  changes  in  a  timely  manner  to  affect  pol¬ 
icy. 

I  Cost  effective 

i _  _ i 

The  number  and  type  of  analytical  tools  should  be  sufficiently  reasonable  not  to  levy  | 
too  high  a  burden  on  those  tasked  with  the  effort.  j 
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Introduction 


Measuring  the  effectiveness  of  military  operations  is  not  a  new  sub¬ 
ject.  Such  measures  have  long  been  developed  and  used,  with  varying 
degrees  of  success,  to  evaluate  progress  during  combat  operations. 

Tracking  the  movement  of  the  forward  edge  of  the  battle  area  is  an 
example  of  a  measure  that  has  been  used  in  previous  combat  opera¬ 
tions  to  provide  part  of  the  picture  regarding  progress  (or  lack  of 
progress).  During  Operation  Desert  Storm,  the  attrition  of  SCUDs, 
tanks,  and  artillery  were  measured  to  assess  progress  in  eliminating 
the  enemy’s  warfighting  capabilities. 

A  familiar,  and  controversial,  measure  was  the  body  bag  count  in  Viet¬ 
nam  [14] .  This  experience  provides  a  caution  regarding  the  care  that 
must  be  taken  in  developing,  choosing,  and  applying  such  measures. 
This  is  all  the  more  true  for  HAOs  because  they  tend  to  be  less  famil¬ 
iar. 

The  past  few  years  have  seen  an  increase  in  the  military’s  involvement 
in  humanitarian  assistance  and  peace  operations.  Measuring  progress 
is  important  in  these  operations  for  the  same  reasons  it  is  in  combat. 
Measures  provide  a  better  picture  of  the  situation,  the  level-of-effort 
being  applied  against  specific  problems,  the  effectiveness  of  these 
efforts,  and,  ultimately,  an  assessment  of  progress  toward  the  desired 
end  state. 

Recent  experience  in  HAOs  has  seen  the  use  and  visibility  of  a 
number  of  measures.  In  these  operations,  many  MOEs  or  other  mea¬ 
sures  were  identified.  During  Operation  Provide  Comfort,  the  com¬ 
mand  tracked  the  number  of  blankets  delivered  to  displaced  persons 
in  the  mountains  in  Northern  Iraq;  during  Operation  Sea  Angel  in 
Bangladesh,  tons  of  food  delivered  was  a  key  measure;  and  in  Soma¬ 
lia,  the  number  of  convoys  escorted  was  one  of  the  measures  used. 
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However,  many  of  the  measures  relied  on  in  past  operations  and  exer¬ 
cises  were  developed  quickly;  consequendy  they  sometimes  lacked 
proper  context.  This  includes  the  examples  cited  above.  All  are  legit¬ 
imate  measures,  but  without  additional  information  they  lack  appro¬ 
priate  context.  The  number  of  blankets  delivered  is  an  indication  of 
a  certain  level-of-effort  being  applied  toward  such  deliveries,  but  it 
doesn’t  address  the  effectiveness  of  this  effort  until  it  is  compared 
with  the  overall  need  for  blankets.  Moreover,  it  was  important  that  the 
immediate  need  for  blankets  (and  seeing  this  reflected  by  corre¬ 
sponding  measures)  not  distract  from  the  accomplishment  of  higher- 
level  objectives,  such  as  the  resettlement  of  displaced  persons  and  ref¬ 
ugees. 

Recent  experience  in  HAOs  has  provided  some  insights  as  to  who 
uses  MOEs  and  related  measures,  how  these  measures  are  being 
developed  and  used,  and  how  well  the  measures  are  serving  the  users. 
In  this  document  we  discuss  several  means  of  organizing  these  mea¬ 
sures  and  criteria  for  considering  their  development  and  application. 
We  also  examine  the  subject  by  reviewing  past  experience  and  using 
a  recent  exercise  that  closely  examined  MOE  development  as  a  case 
study  [14]. 

Purposes  of  measures 

Measures  are  tools.  They  can  provide  a  description  of  ongoing  activi¬ 
ties,  measure  how  well  a  given  level-of-effort  is  succeeding  in  accom¬ 
plishing  certain  specified  tasks,  and  provide  insight  into  the  overall 
progress  of  the  operation. 

Assessing  the  effectiveness  of  the  military’s  efforts  during  an  HAO 
may  be  difficult  because  the  relative  unfamiliarity  of  HAOs,  as  well  as 
vague  objectives  for  a  mission  or  a  lack  of  important  information.  But 
an  assessment  of  operational  effectiveness  is  important  in  order  to 
provide  insights  into  strategies  that  work,  where  assets  may  need  to  be 
shifted,  whether  more  (or  fewer)  forces  are  required,  and  when  the 
end  of  the  operation  (end  state)  has  been  reached. 

MOEs  may  support  different  users,  and  they  can  serve  different  pur¬ 
poses  at  different  levels.  At  the  task  level,  MOEs  can  provide  insights 
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regarding  how  effective  a  given  level-of-effort  has  been  in  addressing 
a  specific  problem.  This  may  help  a  commander  by  highlighting 
problems  in  a  specific  geographic  region,  and  by  providing  a  more 
complete  picture  on  which  to  make  decisions  about  allocating  forces 
to  tasks.  At  a  higher  level,  MOEs  can  assist  in  assessing  progress 
toward  the  end  state.  At  this  level,  measures  can  be  used  by  the  com¬ 
bined/joint  task  force  (C/JTF)  commander  to  address  progress 
toward  the  political  and  policy-level  objectives  set  forth  for  the  mis¬ 
sion,  or  to  address  the  transition  of  responsibilities  to  follow-on  orga¬ 
nizations. 

Examining  MOEs  over  time  can  identify  important  trends  in  the 
operation.  Close  examination  of  these  trends  can  be  used  to  support 
cost-benefit  analysis,  to  show  where  the  most  progress  can  be  made, 
and  to  show  where  additional  investments  of  time  and  effort  may  not 
offer  much  return.  Trend  analysis  can  also  highlight  particular  areas 
of  concern.  An  improving  trend  in  most,  but  not  all,  relief  sectors  may 
warrant  detailed  investigation  and  possibly  a  change  in  the  focus  of 
effort. 

Even  the  process  of  developing  measures  to  assess  effectiveness  can 
serve  an  important  purpose.  This  process  offers  opportunities  to 
engage  the  other  key  HAO  players.  General  MOE  development  can 
be  done  pre-crisis,  and  refinements  based  on  the  specifics  of  the  situ¬ 
ation  and  mission  can  be  done  during  the  planning  phase  of  a  spe¬ 
cific  crisis.  Further  refinements  can  be  made  during  execution.  An 
inclusive  approach  to  MOE  development  affords  important  coordina¬ 
tion  benefits.  It  can  help  to  keep  the  various  components  of  an  HAO 
aligned,  with  all  players  having  a  shared  sense  of  the  mission,  its  objec¬ 
tives,  tasks,  priorities,  and  definitions  of  success. 

Maintaining  an  alignment  between  an  operation’s  political  and 
policy-level  objectives  (which  may  be  revisited  and  updated  during 
the  operation)  and  the  ongoing  military  and  humanitarian  tasks  is 
key  to  preventing  mission  creep.  MOEs  may  be  useful  as  a  tool  in  pre¬ 
venting  mission  creep  by  providing  a  better  picture  Of  how  efforts  are 
being  applied  and  whether  the  objectives  and  tasks  remain  aligned. 
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Road  map 


The  analysis  presented  here  expands  upon  work  started  in  [14]  and 
[15]  in  support  of  research  conducted  for  the  USMC  HAO  study  at 
CNA  and  analysis  conducted  by  the  CNA  representative  at  III  MEF. 
This  study  is  sponsored  by  I  MEF  and  MCCDC.  Its  objective  is  to  iden¬ 
tify  and  analyze  alternative  ways  the  Marine  Corps  might  consider 
using  to  improve  its  ability  to  conduct  HAOs.  Other  analysis  from  this 
study  is  presented  in  [1  -  IB]. 

The  remaining  sections  discuss  criteria  for  developing  measures  for 
assessing  effectiveness,  structuring  HAO  measures,  developing 
MOEs,  and  our  conclusions. 
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Criteria  for  developing  MOEs 

To  develop  measures  to  assess  military  effectiveness  in  HAOs,  some 
criteria  should  be  established.  Various  studies  have  addressed  the  nec¬ 
essary  criteria  [14, 15,  16, 17],  which  are  as  follows: 

•  Mission-related 

•  Comprehensive 

•  Meaningful 

•  Measurable 

•  Sensitive 

•  Timely 

•  Cost  effective 

Mission-related.  By  mission-related  we  do  not  mean  that  the  measures 
should  address  only  the  military  mission  and  the  associated  tasks. 
Measures  for  assessing  effectiveness  should  focus  on  the  overarching 
mission — which  is  the  reason  for  the  intervention — and  not  solely  on 
the  few  specific  military  tasks  assigned  to  the  command  and  the  level- 
of-effort  associated  with  those  tasks.  The  tasks  the  military  conducts 
should  influence  the  humanitarian  situation.  Also,  part  of  the  mission 
eventually  is  to  leave,  therefore  transition  measures  are  needed. 

Comprehensive.  No  single  measure  can  capture  all  of  the  information 
relevant  to  the  humanitarian  mission.  The  complete  list  of  measures 
should  cover  all  aspects  of  the  mission  and  expand  if  the  mission" 
does.  Commands  should  not  focus  solely  on  the  security  aspect  of  the 
mission  (because  the  military  is  most  familiar  with  that  part) ,  nor 
should  it  fail  to  include  measures  that  cover  extra  tasks  (in  the  false 
belief  that  not  acknowledging  new  tasks  prevents  mission  creep) .  The 
different  types  of  measures  should  complement  each  other. 
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Meaningful.  It  is  imperative  that  measures  to  assess  effectiveness  pro¬ 
vide  meaningful  measures  of  progress.  Measures  associated  with  tasks 
should  focus  on  the  effectiveness  of  tasks — not  on  their  accomplish¬ 
ment.  Measures  associated  with  transition  should  focus  on  the  degree 
that  the  follow-on  tasks  are  being  taken  over  by  the  follow-on  organi¬ 
zation.  Measures  associated  with  the  overall  mission  should  focus  on 
the  effectiveness  of  the  military  intervention  on  the  overarching  situ¬ 
ation.  All  five  types  of  measures  should  be  used  together. 

Measurable.  Those  collecting  data  for  measures  should  be  able  to 
assign  consistent  and  accurate  values  to  the  measures.  This  will 
ensure  that  they  are  measured  consistently  over  time  and  space  (to 
see  trends  and  compare  different  areas) ,  and  understood  by  those 
using  the  MOEs  at  all  levels  of  command — especially  at  higher  head¬ 
quarters  distant  from  the  area. 

Sensitive.  The  real  purpose  of  these  measures  is  to  serve  trend  analy¬ 
sis — showing  progress  or  no  progress.  The  measurement  level  should 
provide  enough  detail  to  enable  observers  to  ascertain  whether  the 
situation  is  changing.  Trends  can  be  used  to  support  cost-benefit  anal¬ 
ysis;  to  show  where  the  most  progress  can  be  made;  and  to  show  where 
additional  investments  of  time  and  effort  may  not  offer  much  return. 
Trend  analysis  can  also  highlight  particular  areas  of  concern.  An 
improving  trend  in  most,  but  not  all,  relief  sectors  may  warrant 
detailed  investigation  and  possibly  a  change  in  the  focus  of  effort. 

Timely.  Measures  used  to  assess  effectiveness  should  be  responsive  to 
the  changes  they  are  trying  to  measure  quickly  enough  for  the  com¬ 
mand  to  detect  the  changes  and  act  on  them. 

Cost  effective.  The  number  and  types  of  measures  used  to  assess  effec¬ 
tiveness  should  be  sufficiently  reasonable  that  they  do  not  create  a 
burden  on  those  collecting  and  measuring  them.  These  criteria  must 
be  balanced  against  the  comprehensiveness  of  MOE  criteria. 


Structuring  a  broad  set  of  HAO  measures 

In  this  section  we  describe  types  of  measures  used  to  assess  effective¬ 
ness.  It  shows  how  some  types  parallel  key  elements  of  mission  analysis 
and  how  they  should  be  integrated.  We  then  use  the  types  of  measures 
and  the  components  of  HAOs  to  provide  a  framework  for  developing 
the  measures. 

Using  mission  analysis  to  develop  appropriate  measures 

Mission  analysis  is  a  crucial  part  of  the  planning  process  for  any  oper¬ 
ation.  This  process  is  envisioned  as  beginning  with  the  political  objec¬ 
tives  for  the  operation  being  translated  into  appropriate,  executable 
military  and  humanitarian  tasks.  These  tasks  will  then  require  certain 
capabilities,  and  an  appropriate  force  must  be  tailored  to  provide 
these  capabilities.  This  is  the  model  for  the  mission  analysis  process, 
and  it  serves  very  well  in  ensuring  alignment  of  tasks,  capabilities  and 
forces  with  the  overall  objectives.  It  is  also  very  useful  for  determining 
the  force  that  can  most  efficiently  and  effectively  conduct  the  mission. 
The  value  of  this  model  for  mission  analysis  was  demonstrated  clearly 
during  Operation  United  Shield,  which  supported  the  withdrawal  of 
UN  forces  from  Somalia.  Compressing  or  eliminating  elements  of  this 
process  can  lead  to  misalignments  between  the  objectives  for  the  mis¬ 
sion  and  the  tasks  being  conducted  (which  is  the  principal  factor 
underlying  mission  creep). 

This  translation  of  objectives  to  tasks  to  capabilities  to  forces  is  often 
a  challenging  effort  in  HAOs  and  peace  operations.  The  initial  set  of 
political  objectives  may  be  vague  and  the  implications  of  certain 
objectives  and  their  associated  tasks  may  not  be  readily  apparent. 

The  process  of  translating  political  objectives  into  military  and 
humanitarian  tasks  shapes  the  course  of  the  mission  to  follow.  An 
essential  part  of  mission  analysis  is  identifying  and  communicating 
the  implications  of  the  mission’s  political  objectives  to  the  forces 
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involved  in  executing  associated  tasks.  It  is  also  essential  to  ensure 
that  there  is  an  understanding,  at  the  political  and  policy  levels,  of  the 
potential  implications  associated  with  the  execution  of  certain  tasks. 
Having  thus  determined  the  required  set  of  military  and  humanitar¬ 
ian  tasks,  planners  must  identify  the  capabilities  needed  to  support 
these  tasks  and  tailor  a  force  that  has  these  capabilities. 

These  efforts  during  mission  analysis  are  vital  not  only  to  ensuring 
alignment  between  objectives  and  tasks,  but  also  to  identifying  where 
MOEs  can  be  applied  at  various  levels.  There  is  some  hierarchy  to  the 
set  of  MOEs,  and  here  an  association  is  made  between  key  elements 
of  mission  analysis  and  MOE  development: 

•  Mission-level  MOEs  are  needed  to  address  progress  made 
toward  achieving  the  ultimate  political  objectives. 

•  Task-performance  MOEs  address  specific  military  and  humani¬ 
tarian  tasks. 

•  Level-of-effort  measures  are  needed  at  the  level  of  required 
capabilities  and  forces. 

Level-of-effort  measures  focus  narrowly  on  specific  force  actions. 
Task-performance  MOEs  may  encompass  many  of  the  actions 
described  by  individual  level-of-effort  measures,  placing  these  actions 
into  a  larger  context,  broadening  the  nature  of  the  measure,  and  fre¬ 
quently  indicating  an  action’s  effects  relative  to  total  requirements. 
Mission-level  MOEs  are  broader  still,  encompassing  how  the  neces¬ 
sary  tasks  are  meeting  higher-level  objectives  associated  with  the 
affected  nation’s  overall  security  and  humanitarian  situation. 

Mission-level  MOEs 

At  the  mission-level,  MOEs  should  help  address  progress  toward  the 
political  objectives  set  forth  for  the  mission.  From  recent  operations, 
these  objectives  have  included  providing  a  secure  environment  that 
allows  the  delivery  of  humanitarian  assistance,  promotes  the  redevel¬ 
opment  of  key  institutions,  and  supports  democratic  elections. 
Because  the  objectives  they  address  are  encompassing,  mission-level 
MOEs  must  be  broad.  Specific  examples  of  the  various  types  of  mea¬ 
sures  are  included  later  in  this  paper. 


Task-performance  MOEs 

There  may  be  many  supporting  tasks  associated  with  the  mission’s 
objectives.  MOEs  at  this  next  level  will  support  an  evaluation  of  task- 
performance.  These  task-performance  MOEs  will  be  more  narrowly 
focused  than  the  mission-level  measures.  They  will  address  specific 
actions  taken  to  address  specific  military  and  humanitarian  tasks. 
Many  of  these  measures  compare  the  actions  undertaken  to  address 
a  specific  situation  with  the  total  requirement. 

Level-of-effort  measures 

These  measures  address  individual  actions  taken  by  the  force.  Many 
of  these  are  familiar  measures  that  are  often  widely  cited  in  situation 
reports  (SITREPs)  or  in  media  reports.  Level-of-effort  measures  pro¬ 
vide  insight  into  the  magnitude  of  military  support  to  an  HAO.  Exam¬ 
ples  include  tons  of  food  delivered  or  number  of  convoys  escorted. 
Such  measures  are  not  MOEs  because  they  don’t  measure  effective¬ 
ness,  but  they  often  support  the  measurement  of  broader-level  task- 
performance  or  mission-level  measures. 

Other  types  of  measures 

In  addition  to  the  three  types  of  MOEs  previously  discussed,  two  other 
types  of  measures  are  worth  considering:  transition  measures  and 
general  indicators. 

Transition  measures 

The  military  component  of  an  HAO  or  peace  operation  often  con¬ 
cludes  before  the  humanitarian  or  diplomatic  components.  Also,  the 
U.S.  military’s  involvement  will  end  and  the  operation  will  transition 
to  a  follow-on  UN  force,  the  host-nation  government,  or  the  relief 
organization.  Because  of  these  timing  issues,  the  transition  of  respon¬ 
sibilities  is  an  important  matter.  Transition  measures  provide  insight 
into  progress  on  the  transition  of  the  HAO  to  another  organization. 
Transition  measures  can  be  useful  in  helping  to  assess  the  particular 
responsibilities  assumed  by  follow-on  organizations.  An  effective  tran¬ 
sition  is  important  to  sustain  the  military’s  accomplishments  and  pre¬ 
vent  a  relapse  into  earlier  crisis  conditions. 
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General  indicators 


Certain  general  indicators  may  provide  insights  into  the  current  situ¬ 
ation.  These  indicators,  unlike  MOEs,  are  a  non-quantitative  tool, 
however,  they  can  supplement  MOEs  and  indicate  progress  or  diffi¬ 
culties.  Examples  of  general  indicators  include  food  riots  (indicating 
food  shortages)  or  crowds  returning  to  markets  (indicating  the  pub¬ 
lic’s  perception  of  a  reduced  threat  of  violence). 

Integrating  the  types  of  measures 

As  in  mission  analysis,  the  parallel  types  of  MOEs  need  to  be  exam¬ 
ined  as  a  whole,  not  independently.  In  mission  analysis,  the  political 
and  policy  objectives  are  the  starting  points  for  developing  military 
tasks,  and  the  capabilities  and  corresponding  forces  needed  are 
defined  by  the  tasks.  The  intent  is  that  the  tasks  and  forces  address  the 
political  objectives.  Therefore,  during  operations  the  objectives  and 
associated  tasks  need  to  be  constandy  examined  to  ensure  that  they 
are  complementary.  Similarly,  for  MOEs,  the  task  MOEs  are  devel¬ 
oped  from  the  mission-level  MOEs,  and  the  level-of-effort  measures 
are  a  function  of  the  task  MOEs.  MOEs  can  be  used  as  a  tool  to  eval¬ 
uate  whether  tasks  are  addressing  the  objectives  through  mission- 
level  MOEs.  In  other  words,  the  types  of  MOEs  cannot  be  examined 
in  isolation.  They  must  be  examined  together  in  order  to  understand 
the  dynamic  between  objectives,  tasks,  and  forces. 

Similarly,  transition  measures  must  be  examined  along  with  task  and 
mission-level  MOEs.  Transition  measures  measure  only  whether  the 
transition  is  (or  isn’t)  taking  place,  not  whether  the  follow-on  force  is 
conducting  tasks  to  meet  the  objectives.  Therefore,  transition  mea¬ 
sures  should  be  examined  together  with  task  and  mission-level  MOEs 
to  ensure  that  the  transition  is  effective. 

Characteristics  of  HAOs  for  the  framework 

We’ve  discussed  the  different  levels  of  MOEs  and  how  they  corre¬ 
spond  to  mission  analysis.  The  next  issue  is  to  define  the  framework 
that  can  be  used  to  assess  the  military’s  effectiveness  during  HAOs. 
Our  intent  is  to  provide  a  framework  for  developing  MOEs  that  could 
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expand  or  contract  based  on  the  specific  situation.  We  used  the  dif¬ 
ferent  types  of  measures,  starting  from  mission-level  MOEs  down  to 
level-of-effort  measures  using  mission  analysis,  and  included  transi¬ 
tion  measures  and  general  indicators.  Because  HAOs  can  be  very 
broad  in  scope,  we  then  delineated  the  measures  even  further. 

Based  on  our  analysis  of  past  operations  in  [1  -  IS],  we  found  that 
HAOs  generally  have  some  or  all  of  the  following  components: 

•  Diplomacy 

•  Security 

•  Infrastructure  and  institutions 

•  Public  health 

•  Agriculture  and  economic  situation 

We  won’t  discuss  diplomacy  in  detail  in  this  paper  because  our  focus 
here  is  on  operations.  However,  diplomacy  should  be  recognized  as  a 
key  component  of  HAOs  that  can  influence  operations  on  the 
ground.  It  should  be  conducted  in  parallel  with  the  military  and 
humanitarian  efforts. 

Security  is  often  the  key  reason  for  military  intervention  in  an  HAO 
(i.e.,  to  provide  a  secure  environment).  Providing  security  may  be 
very  broad  in  scope:  providing  protection  for  coalition/U.S.  forces; 
providing  security  so  that  relief  supplies  can  be  delivered;  or  provid¬ 
ing  a  strong  presence  to  prevent  conflict  between  parties,  for  exam¬ 
ple.  In  Restore  Hope,  the  military  provided  security  to  relief  convoys 
and  distribution  centers.  In  Haiti,  military  presence  provided  a  secure 
environment  for  controlling  violence  between  the  previous  dictator¬ 
ship  and  the  incoming  government.  In  Provide  Comfort,  the  military 
provided  security  so  that  the  Kurds  could  return  to  their  homes. 

Infrastructure  and  institutions  are  also  other  common  components  of 
HAOs.  The  military  may  be  asked  to  rebuild  institutions,  such  as  the 
local  police  force,  a  court  system,  schools,  and  so  forth.  In  the  case  of 
Haiti,  the  military  was  supporting  the  new  Haitian  government.  Part 
of  an  HAO  could  include  repairing  or  replacing  infrastructure,  such 
as  sewage  and  water  systems;  clearing  debris;  and  restoring  public 
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works.  For  example,  in  Restore  Hope  the  military  built  bridges  and 
repaired  roads  to  support  the  overall  mission  of  getting  relief  supplies 
to  those  who  needed  them.  And  in  Bangladesh,  during  Sea  Angel,  the 
military  cleared  debris  allowing  renewed  access  to  key  elements  of  the 
infrastructure  to  support  the  Bangladesh  government. 

Public  health  is  also  a  key  component  of  HAOs  and  this  may  be  one 
of  the  reasons  the  U.S.  Government  is  asked  to  intervene  in  the  first 
place.  Public  health  reflects  the  general  health  of  the  population, 
which  is  influenced  by  disease,  starvation,  and  lack  of  sanitation. 
Almost  all  HAOs  with  U.S.  military  intervention  have  a  public-health 
component,  including  Restore  Hope,  Provide  Comfort,  Sea  Angel, 
and  others. 

Finally,  the  agriculture  and  economic  situation  can  be  a  major  factor 
in  an  HAO.  The  economy  may  be  destroyed  by  environmental  factors 
affecting  the  food  supply;  or  conflict  may  be  preventing  farmers  from 
cultivating  their  land  which  was  the  case  in  Restore  Hope. 

Defining  the  framework:  Putting  the  pieces  together 

All  or  some  of  these  general  components  have  characterized  past 
HAOs,  as  the  examples  show.  Using  these  components,  along  with  the 
different  types  of  measures,  provides  a  framework  for  developing 
measures  of  effectiveness.  Table  2  illustrates  this  framework. 


Table  2.  Framework  for  developing  measures 


Characteristics/ 

Types 

Security 

Institution/ 

Infrastructure 

Public 

Health 

Agriculture/ 

Economic 

Mission-level  MOEs 

Task  MOEs 

Level-of-effort 

measures 

Transition  measures 

General 

Indicators 

| 
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When  developing  MOEs  using  this  framework,  the  MOEs  defined  by 
the  components  and  types  of  MOEs  should  not  be  examined  in  isola¬ 
tion.  These  characteristics  are  often  interrelated  or  inextricably 
linked.  For  example,  the  military  may  be  asked  to  provide  security  for 
transporting  medical  supplies.  These  related  characteristics  affect  the 
security  situation,  as  well  as  the  public-health  situation.  These  tasks,  in 
turn,  may  affect  the  overall  policy  objectives  of  improving  security  and 
decreasing  the  death  rate  due  to  disease. 

The  next  section  provides  an  application  of  this  framework  that  can 
be  used  when  developing  and  using  measures  before,  during,  and 
after  an  HAO.  Our  intent  is  to  provide  a  starting  point  and  a  few 
examples  that  illustrate  how  to  develop  measures  for  a  mission. 


Developing  HAO  measures 

The  previous  section  described  a  framework  for  developing  HAO 
measures.  This  section  presents  the  evolution  of  MOE  development 
relative  to  the  phases  of  an  operation  and  an  example  for  developing 
HAO  measures.  This  example  is  certainly  not  inclusive  for  all  HAOs. 
The  measures  need  to  be  developed  on  the  basis  of  the  particular  sit¬ 
uation.  The  intent  here  is  only  to  provide  a  starting  point  for  the  mil¬ 
itary. 

Measures  and  the  phases  of  an  operation 

Pre-crisis 

Development  of  candidate  MOEs  can  begin  before  any  operation  is 
imminent  or  any  crisis  action  planning  is  initiated.  The  pre-crisis 
phase  offers  an  opportunity  for  careful,  deliberate  examination  of 
potential  types  of  missions;  coordination  with  other  key  players 
involved  in  humanitarian  assistance  and  peace  operations;  and  for 
MOE  testing  during  exercises. 

Crisis  indicators — for  example,  the  onset  of  drought  conditions  or  an 
increase  in  factional  fighting — can  be  used  to  identify  regions  where 
U.S.  intervention  may  be  required.  These  crisis  indicators  are  another 
type  of  measure,  and  the  information  collected  to  develop  these  indi¬ 
cators  (and  particularly  information  derived  from  more-formal  assess¬ 
ments  made  as  a  crisis  worsens)  may  provide  an  important  baseline 
against  which  later  measures  can  be  compared.  Often,  the  relief  orga¬ 
nization  or  the  Center  for  Disease  Control  make  mission-level  mea¬ 
surements,  particularly  in  the  area  of  public  health,  before  military 
intervention  is  imminent.  These  can  be  used  as  crisis  indicators. 
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Planning  for  a  specific  crisis 

At  some  point,  certain  crises  will  reach  the  point  where  specific  oper¬ 
ational  planning  begins.  As  previously  discussed,  there  are  natural 
parallels  between  certain  types  of  measures  and  key  elements  of  mis¬ 
sion  analysis.  As  planners  proceed  through  the  planning  process,  they 
should  refine  the  more  general  measures  identified  during  the  pre¬ 
crisis  period,  tailoring  the  measures  to  the  specific  objectives 
required  at  the  mission  level,  the  military  and  humanitarian  tasks  to 
be  performed,  and  the  application  of  effort  against  these  tasks. 

Developing  and  choosing  the  appropriate  measures  can  be  a  valuable 
element  of  the  planning  process.  Because  of  the  parallels  with  mis¬ 
sion  analysis,  determining  the  set  of  measures  can  be  a  useful  way  of 
ensuring  that  objectives,  tasks,  and  capabilities/forces  are  aligned, 
and  that  these  have  been  identified  to  a  sufficient  level  of  detail.  It 
also  offers  opportunities  to  engage  the  other  key  players  in  the  oper¬ 
ation,  including  relief  organizations,  coalition  forces,  and  the  UN  (if 
possible),  to  frame  the  full  set  of  objectives  for  the  mission,  the  full 
range  of  tasks  that  will  be  conducted,  and  the  application  of  effort  to 
specific  tasks.  Experience  with  previous  “joint”  MOE  development 
suggests  that  it  can  be  an  effective  coordination  mechanism. 

During  the  planning  phase,  some  initial  information  should  be  avail¬ 
able  from  a  range  of  sources  on  the  situation  in  country.  As  noted, 
such  information  and  assessments  are  important  because  they  can 
allow  a  baseline  to  be  developed  for  later  comparisons. 

Mission  execution 

During  mission  execution,  the  set  of  measures  can,  and  at  times 
should,  be  further  refined.  This  phase  is  also  where  most  of  the  actual 
measurements  will  take  place. 

Measures  should  be  refined  as  the  nature  of  the  mission  and  its  objec¬ 
tives,  tasks,  and  priorities  change.  These  changes  occur  as  improved 
information  becomes  available  on  the  initial  situation,  and  also  as  the 
situation  changes  over  time  (e.g.,  in  Provide  Comfort,  measuring  how 
well  efforts  were  proceeding  to  provide  shelter,  blankets,  etc.  in  the 
camps  wasn’t  relevant  as  the  mission  proceeded  into  the  repatriation 


of  displaced  persons  and  refugees  phase) .  As  noted,  the  measures 
(and  the  process  associated  with  developing,  choosing,  and  applying 
the  measures)  may  provide  a  tool  for  ensuring  continued  alignment 
among  political  objectives,  military  and  humanitarian  tasks,  and  spe¬ 
cific  efforts.  The  risk  of  mission  creep  is  thus  decreased. 

In  performing  the  actual  measurements,  there  will  be  a  number  of 
sources  of  information.  Military  forces  routinely  collect  some  of  this 
information,  but  some  of  the  measures  will  require  additional  collec¬ 
tion.  The  host  nation,  key  organizations  within  the  host  nation,  and 
relief  organizations  are  also  likely  to  be  key  sources  of  information 
that  will  support  these  measurements. 

Transition 

During  the  transition  phase,  the  responsibility  for  tasks  currently 
being  accomplished  by  departing  forces  (which  are  still  required 
during  subsequent  operations)  must  be  transitioned  to  another  orga¬ 
nization.  Transition  measures  allow  for  a  clear  representation  of  the 
fraction  of  these  tasks  that  have  been  handed  to  follow-on  organiza¬ 
tions.  Examples  include  the  fraction  of  required  airfields  turned  over 
to  a  follow-on  force’s  control  or  the  fraction  of  major  supply-route 
maintenance  turned  over  to  the  host  nation  or  to  a  contractor. 

Later  rehabilitation  and  redevelopment  efforts  (post-military 
involvement) 

Because  other  key  players  may  continue  operating  in  country  long 
after  the  military  component  of  the  mission  reaches  an  end,  it  is 
important  to  consider  the  later  requirements  of  rehabilitation  and 
redevelopment.  Although  the  military  will  not  be  involved  in  these 
phases,  it  may  be  able  to  help  set  conditions  that  allow  operations  to 
proceed  successfully  into  these  phases.  A  key  consideration  associated 
with  transition  is  ensuring  that  accomplishments  are  sustainable  into 
these  later  phases.  Measures  can  provide  some  input  regarding 
accomplishments,  their  sustainability,  and  their  scope  compared  to 
the  necessities  of  rehabilitation  and  redevelopment. 
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An  example 


We  are  providing  an  example  for  no  particular  scenario  other  than 
some  form  of  military  response  to  a  humanitarian  situation,  be  it 
purely  humanitarian  or  part  of  a  larger  effort,  such  as  peacekeeping. 
We  present  our  examples  using  the  framework  we  have  defined. 
Because  of  space  considerations,  we  have  divided  the  measures 
according  to  the  components  of  an  HAO:  security,  infrastructure  and 
institutions,  public  health,  and  agriculture  and  economic  situation. 
However,  because  the  measures  are  interrelated  between  the  compo¬ 
nents  and  the  types  of  measures,  they  should  be  examined  as  a  whole. 


Security 


Table  3  provides  examples  of  security  measures  for  each  of  the  five 
previously  discussed  types  of  measure.  Because  the  military  is  the  pri¬ 
mary  force  to  provide  security,  it  would  be  responsible  for  collecting 
the  data  for  developing  the  measures.  In  some  cases,  relief  organiza¬ 
tions  may  have  some  information  to  contribute  to  the  measure-devel¬ 
opment  effort,  such  as  the  fraction  of  the  whole  inventory  stolen  from 
warehouses. 


Table  3.  Example  of  security  measures 


Type 

Examples  of  specific  measures 

1  Mission-level  MOEs 

j 

i 

i 

! 

i 

1 .  Death  rate  due  to  violent  cause 

2.  Fraction  of  total  traffic  needing  no  security 

3.  OJTF  casualty  rate  (by  contingent  as  a  fraction  of  j 
that  coalition  partner's  total  force) 

Task-performance  MOE  \  1 .  Area  patrolled  divided  by  total  area  requiring  j 

j  patrolling 

2.  Convoys  escorted  divided  by  the  convoy  escort 
|  requirement 

Level -of-effort  measures  j  1 .  Number  and  size  of  patrols 

2.  Number  of  convoys  escorted,  and  size  of  forces 
assigned  to  convoy  escort 

Transition  measures 

1 .  Fraction  of  security  requirements  assumed  by 
host  nation  or  follow-on  forces 

General  indicators 

. 

1 .  Food  riots 

2.  Crowds  in  markets 

3.  Human-rights  violations 

22 


Infrastructure  and  Institutions 


Public  health 


Table  4  provides  an  example  of  infrastructure  and  institution  mea¬ 
sures  for  each  type  of  measure.  Infrastructure  and  institutions  make 
up  another  area  that  has  required  major  tasking  for  the  military  in 
past  operations.  Thus,  the  military  will  probably  need  to  collect  the 
data  for  developing  measures,  particularly  at  the  task-performance 
level.  Some  of  the  mission-level  data  needed  for  measure  develop¬ 
ment  may  be  available  through  relief  organizations  or  in  other  assess¬ 
ments  that  are  routinely  conducted  for  HAOs,  such  as  the  disaster- 
assistance-response-team  survey. 


Table  4.  Example  of  infrastructure  and  institutions  measures 


Mission-level  MOEs 

1 .  Fraction  of  local  institutions  able  to  meet  local 
needs 

2.  Percentage  of  areas  where  required  basic  services 
can  be  provided 

3.  Fraction  of  displaced  persons  and  refugees  repatri¬ 
ated  to  their  homes 

Task-performance  MOEs 

1 .  Number  of  police  divided  by  number  required 

2.  Number  of  courts  divided  by  number  required 

3.  Fraction  of  required  airfields,  ports,  and  major 
supply  routes  opened  to  traffic 

4.  Fraction  of  required  food,  potable  water,  sanita¬ 
tion,  etc.  being  provided 

Level-of-effort  measures 

1 .  Number  of  trained  police 

2.  Number  of  visual  flight  rules-capable  airfields 
being  operated 

3.  Number  of  miles  of  roads  improved  to  desired 
condition 

Transition  measures 

Fraction  of  required  responsibilities  (running  ports  or 
airfields,  maintaining  roads,  providing  lift,  and  so 
forth)  turned  over  to  the  host  nation,  contractors, 
follow-on  forces,  and  so  forth. 

General  Indicators 

Elections 

Table  5  provides  some  examples  of  public-health  measures  for  each 
type  of  measure.  Most  mission-level  measures  are  collected  by  relief 
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organizations  in  the  host  nation  or  by  the  Center  for  Disease  Control 
(CDC) .  The  data  to  calculate  the  other  measures  would  need  to  be 
collected  by  the  military  in  conjunction  with  relief  organizations  and 
the  host  nation. 


Table  5.  Example  of  public-health  measures 


Type 

Examples  of  specific  measures 

Mission-level  MOEs 

1 .  Crude  mortality  rates 

2.  Starvation  rate 

3.  Cause-specific  death  rates 

Task-performance  MOEs 

—  .  _ . . 

1 .  Fraction  of  supplies  delivered  vs.  required 

2.  Fraction  of  sanitation  services  provided  vs. 
required 

Level-of-effort  measures 

! 

1 .  Number  of  supplies  delivered 

2.  Tons  of  food  delivered 

3.  Gallons  of  water  provided 

Transition  measures  : 

i  ; 

1  i 

1 .  Fraction  of  requirements  being  met  by  the 
follow-on  organization  i 

General  indicators 

! 

1 .  No  walking  skeletons 

2.  No  visible  corpses  i 

Agriculture  and  economic  situation 

The  military  may  not  be  asked  to  intervene  in  the  area  of  agriculture 
because  the  relief  community  usually  addresses  those  requirements. 
In  addition,  the  military  is  not  trained  to  address  agriculture  needs. 
However,  it  can  perhaps  influence  the  agriculture  situation  by  provid¬ 
ing  security  for  relief  convoys,  providing  a  sufficiently  secure  environ¬ 
ment  for  people  to  go  to  markets,  and  so  forth.  The  military  may  also 
have  a  direct  impact  on  the  economic  situation.  The  local  population 
can  be  employed  by  the  military  to  conduct  humanitarian  tasks,  such 
as  cleaning  up  and  distributing  food.  In  addition,  if  the  military 
begins  to  rebuild  institutions,  it  is  creating  a  potential  source  of 
employment  by  the  local  government  or  business. 

Table  6  shows  some  examples  of  agriculture  and  economic  measures 
for  each  type  of  MOE.  Again,  these  measures  should  not  be  examined 
in  isolation,  but  as  a  whole  among  the  types  and  categories.  Much  of 
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the  data  for  the  measures  will  be  collected  by  relief  organizations  or 
the  host  nation,  particularly  the  mission-level  MOEs.  Some  of  the  data 
for  the  other  measures  would  need  to  be  collected  by  the  military. 


Table  6.  Examples  of  agriculture  and  economic  measures 


Type 

Examples  of  specific  measures 

Mission-level  MOEs 

1 .  Fraction  of  land  cultivated  vs.  total  arable  land 

2.  Fraction  of  population  having  no  food-aid 
requirement  vs.  total  population 

3.  Fraction  of  population  able  to  grow  or  buy  food 

4.  Employment  rate 

Task-performance  MOEs 

1 .  Fraction  of  seeds  or  tool  kits  provided  vs.  need 

2.  Fraction  of  population  employed  by  military  in 
food  for  work  programs 

Level-of-effort  measures 

1.  Number  of  seed/tool  kits  provided 

2.  Number  of  people  employed  by  the  military  in 
food-for-work  programs 

Transition  measures 

1 .  Fraction  of  population  transitioning  from  food 
for  work  to  permanent  employment 

General  indicators 

1 .  Market  price  of  food 

2.  Types  of  food  in  market 
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Choosing  good  measures  and  measuring  them  well  are  necessary 
actions,  but  they  are  not  sufficient  to  ensure  successful  measuring 
efforts.  Measures  to  assess  effectiveness  must  also  be  used  appropri¬ 
ately.  They  are  not  a  cure  for  the  problems  of  military  intervention  in 
HAOs.  MOEs  have  several  potential  dangers  that  are  related  to  the 
criteria  for  MOE  development: 

•  Collecting  the  data  for  and  analyzing  the  measures  may 
become  costly  in  terms  of  time  and  effort  spent. 

•  Viewing  one  measure  as  the  “ultimate”  measure  and  the  key  to 
HAO  success  excludes  many  other  possible  indicators  of 
progress. 

•  Using  biased  or  inaccurate  data  to  develop  measures  can  inap¬ 
propriately  influence  whether  the  military  continues  a  mission. 
Some  factors  are  outside  the  military’s  control,  such  as  the 
weather  or  the  effectiveness  of  other  militaries,  the  host  nation, 
UN  agencies,  and  relief  organizations  that  can  affect  the  mea¬ 
sures.  The  data  used  for  developing  the  measures  must  be 
examined  carefully  for  bias. 

Given  these  potential  problems  when  using  MOEs,  the  job  of  the 
commander  is  to  place  the  proper  emphasis  on  MOEs  and  to  put 
them  in  the  proper  context  so  that  they  are  used  correctly.  When 
MOEs  are  used  properly,  they  will  provide  a  more  sophisticated 
understanding  of  the  humanitarian  situation  and  the  military  opera¬ 
tion.  They  also  can  be  used  to  improve  the  effectiveness  of  the  mili¬ 
tary’s  efforts  in  an  HAO. 
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